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ADAPTIVE FILTER FOR 
NETWORK ECHO CANCELLATION 

FIELD OF THE INVENTION 

This invention relates generally to the field of signal processing and in particular 
to adaptive filtering, specifically that which is well suited for network echo cancellation 
applications in a telephone network. 

BACKGROUND OF INVENTION 

Adaptive filters operate on a supplied signal in a prescribe manner such that a 
desired output signal is generated. Typically, adaptive filters generate a transfer function 
according to an algorithm that includes updating of the transfer function characteristic in 
response to an error signal. In this manner, the filter characteristic is optimized to 
produce a desired result. 

When used in an echo canceller, an adaptive filter is used to generate an echo path 
estimate that is updated in response to an error signal. Echoes commonly occur because 
of imperfect coupling of incoming signals at the 4-to-2 wire junctions in communications 
systems. The echoes typically result because the impedance of the 2 -wire facility is 
imperfectly balanced in the 4-to-2 wire junction, causing the incoming signal to be 
partially reflected over an outgoing path to the source of incoming signals. 

Adaptive echo cancellers have been employed to mitigate the echoes by adjusting 
the transfer function (impulse response) characteristic of an adaptive filter to generate an 
estimate of the reflective signal or echo and, then, subtracting it from the outgoing signal. 
The filter impulse response characteristic and, hence, the echo estimate is updated in 
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response to continuously updated samples of the outgoing signal for more closely 
approximating the echo to be cancelled. 

Additionally, double talk detectors (DTD) are generally used to disable adaptation 
during double talk conditions, that is when both the near end and far end party to a 
telephone conversation taking place across a telephone line speak simultaneously. 
Because the double talk detector cannot disable adaptation at the precise instant the 
double talk occurs, a number of samples occur in the delay period between the 
commencement of actual double talk and the double talk detector shutting down 
adaptation. Samples taken during this delay period can and often do perturb the echo 
path estimate considerably. Also, characteristic changes in the system due to 
environmental or other causes can also perturb the filtering. 

Thus, although prior art arrangements of adaptive filters perform satisfactorily in 
some applications, often it is impossible to simultaneously achieve both sufficiently fast 
response and the ability to resist perturbations caused by samples occurring prior to the 
cessation of adaptation. 

SUMMARY OF THE INVENTION 

The present invention overcomes shortcomings in the prior art. Specifically, the 
present invention is based upon a recognition that fast converging algorithms such as 
normalized least mean squares (NLMS), proportional normalized least mean squares 
(PNLMS) and a version which combines the two, specifically PNLMS++, are insufficient 
of themselves to resist perturbations in the echo path estimate resulting from disruptive 
samples collected in the echo canceller prior to the cessation of adaptation by the double 
talk detector. In addition to the fast conversion algorithms just discussed, there is 
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another, known an affine projection algorithm (APA). In one inventive aspect of the 
present invention, a proportional affine projection algorithm (PAPA) is presented which 
combines APA and PNLMS++. This algorithm converges very fast, and unlike 
PNLMS++, is not as dependent on the assumption of a sparse echo response. 

Yet, all of the above fast converging algorithms can benefit from improved 
robustness so as to yield the desired result of fast convergence while at the same time 
achieving decreased sensitivity to near-end disturbances. To that end, the present 
invention incorporates the addition of an adaptive, scaled non-linearity to the echo 
canceller to improve its robustness, that is its resistance to near-end disturbances. The 
adaptive scaled non-linearity, which improves robustness, may be combined with any of 
the aforementioned fast converging algorithms. This yields robust versions of the 
algorithms that when applied by appropriate system components results in a filter that can 
be used in an echo canceller and that maintains good convergence rates for the echo 
canceller while decelerating the divergence due to undetected double talk. 

Other objects and features of the present invention will become apparent from the 
following detailed description, considered in conjunction with the accompanying drawing 
figures. It is to be understood, however, that the drawings, which are not to scale, are 
designed solely for the purpose of illustration and not as a definition of the limits of the 
invention, for which reference should be made to the appended claims. 


In the drawing figures, which are not to scale, and which are merely illustrative, 
and wherein like reference numerals depict like elements throughout the several views: 


BRIEF DESCRIPTION OF THE DRAWINGS 
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FIG. 1 is a simplified block diagram of an echo canceller suitable for utilizing the 
proportional affine projection algorithm (PAPA) disclosed herein; 

FIG. 2 is a detailed block diagram showing the block diagramatic representation 
of where in the echo canceller of the present invention the scaled non-linearity device 
should be inserted; 

FIG. 3 is an alternative representation of the echo canceller of the present 
invention; 

FIG. 4 is a graphical representation of a preferred scaled non-linearity which may 
be applied to the echo canceller of the present invention to improve robustness thereof; 

FIG. 5 is a tabular example of the computational requirements for the scaled non- 
linearity; 

FIGS. 6a and b are graphical representations of the impulse response and 
magnitude of the frequency response, respectively, of an exemplary hybrid utilized for 
performance simulations of the present invention; 

FIG. 7 is a group of graphs indicating performance results of a first embodiment 
of the present invention; 

FIG. 8 is a further group of graphs indicating performance results of the first 
preferred embodiment of the present invention; 

FIG. 9 is a further group of graphs indicating performance results of the first 
preferred embodiment of the present invention; 

FIG. 10 is a further group of graphs indicating performance results of the first 
preferred embodiment of the present invention; 
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FIG. 1 1 is a further group of graphs indicating performance results of the first 
preferred embodiment of the present invention; 

FIG. 12 are graphical representations of divergence and convergence comparisons 
utilizing differing algorithms in a first embodiment of the present invention; 

FIG. 13 are further graphical representations of divergence and convergence 
comparisons utilizing differing algorithms in a first embodiment of the present invention; 

FIG. 14 is a further group of graphs indicating performance results of the first 
preferred embodiment of the present invention; 

FIG. 15 is a group of graphs indicating performance results of a second 
embodiment of the present invention; 

FIG. 16 is a group of graphs indicating performance results of a second 
embodiment of the present invention; 

FIG. 17 is a group of graphs indicating performance results of a second 
embodiment of the present invention; 

FIG. 18 is a group of graphs indicating performance results of a second 
embodiment of the present invention; 

FIG. 19 is a group of graphs indicating performance results of a second 
embodiment of the present invention; 

FIG. 20 is a group of curves depicting the impulse response and corresponding 
magnitude of frequency responses for three alternate test hybrid configurations used in 
performing tests on the second embodiment of the present invention; 

FIG. 21 is a group of graphical representations of the signal utilized for a test 3 A 
performed on a second embodiment of the present invention; 
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FIG. 22 is a group of graphs indicating performance results of a first test 
performed on a second embodiment of the present invention; 

FIG. 23 is a further group of graphs indicating performance results of a first test 
on a second embodiment of the present invention; 

FIG. 24 is a further group of graphs indicating performance results of a first test 
on a second embodiment of the present invention; 

FIG. 25 is a further group of graphs indicating performance results of a first test 
on a second embodiment of the present invention; 

FIG. 26 is a further group of graphs indicating performance results of a first test 
on a second embodiment of the present invention; 

FIG. 27 is a group of graphical signal representations for a group of signals used 
for a second test of a second embodiment of the present invention; 

FIG. 28 is a group of graphs indicating performance results of a second test 
performed on a second embodiment of the present invention; 

FIG. 29 is a further group of graphs indicating performance results of a second 
test performed on a second embodiment of the present invention; 

FIG. 30 is a further group of graphs indicating performance results of a second 
test performed on a second embodiment of the present invention; 

FIG. 3 1 is a further group of graphs indicating performance results of a second 
test performed on a second embodiment of the present invention; and 

FIG. 32 is a further group of graphs indicating performance results of a second 
test performed on a second embodiment of the present invention. 
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DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

With initial reference to FIG. 1, which is a schematic diagram of a preferred echo 
cancelling system in accordance with the present invention, there is shown, in a dashed 
line box, an echo canceller 1 forming a part of a network telephone circuit 400 which is 
connected via digital-to-analog (D/A) converter 90 to an analog hybrid 100 which 
connects, via local loop 110, to a telephone line or other telephone instrument 120 at a 
near end 500 in a manner known in the art. Communications from the near end pass 
through local loop 110 and through hybrid 100 through an analog-to-digital converter 98 
for communication to a far end, represented generally at 200. Situated across the network 
circuit, in parallel to the hybrid 100, is a fast impulse response (FIR) filter 10 connected 
through summing node 20. In a manner known in the art, fast impulse response filter 10 
attempts to duplicate the impulse response of any discontinuity at the hybrid connection 
point, thereby cancelling echoes. Fast impulse response filter 10 is an adaptive filter 
which receives its filter coefficients, at each sample interval, from coefficient vector 
update device 40. Coefficient vector update device 40 receives as it input an echo signal 
e nj which is a residual echo signal, a step size variable [i, 42, which, in a manner known 
in the art, effects the speed of convergence, and an excitation signal x n generally received 

from the far end 200. 

A double talk detector 30 is also part of echo canceller 1 for controlling 

coefficient vector update device 40 by deactivating coefficient vector update device 40 

during the occurrence of double talk. This effectively halts the adaptation taking place in 

fast impulse response filter 10. There is, however, a finite time delay between the actual 

onset of double talk and the effective discontinuation of adaptation in fast impulse 
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response filter 10. Thus, while it is the goal of double talk detector 30 to in effect 
disconnect coefficient vector update device 40 from the circuit upon the detection of 
double talk, typically by setting the step size 42) to zero, in actuality instantaneous 
response is presently unobtainable. Thus, the finite time that the double talk detector 30 
needs to detect double talk is often enough to misalign the filter coefficients of coefficient 
vector update device 40 considerably. In other words, even a small error in the double 
talk detector 30 leads to relatively large errors in the adaptive coefficients provided to fast 
impulse response filter 10 by coefficient vector update device 40. Thus, the samples of 
echo signal e n obtained in coefficient vector update device 40 during the period of 
undetected double talk leads to significant misalignment of fast impulse response filter 10 
which, upon activation of double talk detector 30 and the discontinuance of adaptation, 
remains frozen in a significantly non-adapted or divergent state. Under such 
circumstances, effective suppression of residual echo cannot be obtained. 

Thus, in order to render echo canceller 1 more robust, that is more able to 
decelerate the divergence due to undetected double talk while maintaining good 
convergence for the echo canceller, an adaptive scaled non-linearity device 50, as seen in 
FIG. 2, is inserted in echo canceller 1 for application of an adaptive scaled non-linearity 
to error signal e n so as to reduce the impact of undetected double talk while maintaining 
fast convergence. With reference to FIG. 4, which will be discussed further below, a 
preferred adaptive scaled non-linearity 300 is graphically depicted. 

With reference now to FIG. 3, there is shown an alternate representation of echo 
canceller 1 in accordance with the' present invention showing double talk detector 30 
controlling coefficient vector update device 40 which generates filter coefficients for use 
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by fast inpulse response filter 10 by utilizing an adaptive algorithm incorporating the 

adaptive scaled non-linearity (e.g. FIG. 4) fed by adaptive scaled non-linearity device 50.. 

True echo path 12 is representationally shown connected to the echo canceller through a 

representational summing node 22, at which point v n is shown as an input signal which 

includes background noise and possibly near-end signals from near end 500. 

As mentioned above, it is preferable for the coefficient vector update device 40, 

which provides the adaptive filter coefficients utilized by the adaptive filter FIR 10, to 

provide for fast convergence through the use of fast converging algorithms. As used 

herein, the term coefficients of the adaptive filter, adaptive filter weights, filter taps and 

taps weights, shall be used interchangeably. Further, the following notations will be used 

consistently throughout herein: 

x n = Far-end signal, (1) 

v n = Background noise and possibly near-end signals, (2) 

y n = Echo and background noise possibly including (3) 
near-end signal, 

Y n = [y n . . . y n - p +]] T 9 Vector of samples y n (4) 

x n = [x n . . . x n _i+ /] T , Excitation vector, (5) 

X n = [x n . . . x n „p+ /], Excitation matrix, 
(6) 

h ep = [h Q , ...hi-l], True echo path, (7) 

h n = [h(),n>- • • 9 hL-l,n\ > Estimated echo path, (8) 

A A A 

h n = [h Q n, h L-l,n\ T > A 11 alternative coefficient (9) 
vector for fast APA, 

e n = yn-h T n x n> Residual echo, (10) 
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e n = Y n - X T n hn, Residual echo vector, (11) 

z\ n = fth element in any arbitrary vector z n . (12) 
As used herein, L is the adaptive filter length and p is projection order of the APA. 
As discussed further below, only the preferred exemplary projection order p = 2 is 
considered, it being recognized by the person of skill that other projection orders may be 
considered and utilized, as a matter of application specific design choice. 

NLMS, PNLMS and PNLMS++ 

As mentioned above, it is preferred to use fast converging algorithms to drive FIR 
10. Such algorithms may be, for example, NLMS and PNMLS, as set forth below: 

h n +i= h n + -A jwfc, NLMS (13) 

X fJ Xn+S 

h n+ l= h n + — ^ GnXnen. PNLMS (14) 

X n GnXn + 5 

where 

G n = diag{gbj U ... 9 gt.\ tn } 9 (15) 
is a diagonal matrix which adjusts the step-sizes of the individual taps of the filter 10, ju is 
the overall step-size parameter, and S is a regularization parameter which prevents 
division by zero. The diagonal elements of G n are calculated as follows 

L max = max{6 P) |/k>,» - i,| , . . ., \hi-ui~\ }, (16) 
Yi = max{pL max , | h^ | }, 0 < i < L - 1, (17) 

Li = i=2>, (18) 
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g\ ~ Yi/Li, 0 < i < L- 1 ? (19) 

Parameters dp and p are positive numbers with typical values bp =0.01, p=5/L. 
While these known fast converging algorithms are suitable in certain applications, 
an improvement over these two algorithms has been developed by personnel of the 
assignee called PNLMS++, which is set forth in An Efficient, Fast Converging Adaptive 
Filter For Network Echo Cancellation , published at Ascilomar Conference on Signal 
Systems and Computers, Nov. 1-4, 1998, Pacific Grove, Ca., which in essence combines 
NMLS and PNMLS by utilizing one or the other in alternate time steps in the matrix G w , 
as set forth below: 

hn+1 = h n +——^ Gnxne*, PNLMS++ (20) 

X n GnXn + 5 

G n = diag{gQ n .~,gL-l,n}> diagonal individual step-size matrix, (21) 
ju is the overall step-size parameter and 5 is the regularization parameter that prevents 
division by zero. For odd-numbered samples the diagonal entries of the step-size matrix, 
G n are calculated as follows, which yields PNLMS; 

h max = max{5?,|/20,/i-i|,...,|/zL-i t «-i|}, (22) 

Yi,n = iwax{/?/imax,|/ii,«-i|},0<x>L-l, (23) 

L-\ 

Si,n = y i)n /Yr^<i<L-\. (24) 

1=0 

For even samples, 

G n = I (25) 
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which results in an NLMS step. Parameters Sp - 0.01, and pare positive numbers with 


typical values Sp = 0.01, p= 51 L. 


PAPA 


Another type of known fast converging algorithm is the Affine Projection 
Algorithm (APA), which is a special case of NLMS, but recognizably faster. This may be 
combined with the proportional step size of PNLMS to yield a new, improved fast 
converging algorithm referred to herein as Proportional Affine Projection Algorithm 
(PAPA). This new algorithm is set forth below as: 


where G n is as defined in the last section and (X T n X n +8i) l , is an estimate of the inverse 
correlation matrix of the input signal, 


This matrix "whitens" the input data, X n , and thus the convergence rate of the 
adaptive filter is increased. With G n = 1 , equation (26) reduces to the standard, known 
APA. As evident, and as mentioned above, PAPA is obtained by combining APA with 
the proportional step-size of PNLMS++. Note, however, that it is preferable to omit the 
matrix G n in the definition of R^n to save computations. Inclusion of the matrix requires 
an additional 4L multiplications per sample, but it is believed that the effect on filter 
performance is minimal. Thus omission or inclusion of the matrix is an application 
specific matter of design choice. 
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Additionally, the choice of order of projection in the APA is governed by the 
properties of the input signal. For speech, an order of 6 to 10 would seem appropriate 
since it is known in the art that short term properties of speech are well modeled by an 
auto regressive process of that order. However, it is also known that choosing a low 
value for the order (e.g. 2) yields a practical algorithm with low complexity yet fast 
convergence rate. With an order of 2, it is also easy to maintain the numerical stability of 
the algorithm. 

As seen in FIGS. 1 and 3, a double talk detector (DTD) 30 is a part of echo 
canceller 1 and disables coefficient vector update device 40, albeit imperfectly. By way 
of example, a simple and efficient way of detecting double-talk in DTD 30 is to compare 
the magnitude of the far- and near-end signals and declare double-talk if the near-end 
magnitude becomes larger than a threshold value set by the far-end signal, (1). That is, if 

|>vj| > ~ max { |jc« |, |x» - 1|, . . . , |jt» - l + i |, } (28) 

double-talk is declared. A so called hangover time, Thold IS ^ so included which means 
that if double-talk is detected, adaptation of the filter 10 is inhibited for this pre-specified 
duration beyond the end of double-talk. 

R-NLMS, R-PNLMS and R-PNLMS++ 

However, and as mentioned above, there is a finite period of time from the 
commencement of double-talk and the point when DTD 30 disables adaptation in 
coefficient vector update device 40. It is therefore desirable to make the aforementioned 
fast converging algorithms robust, i.e resistant to near end disturbances, whether 
introduced by undetected double talk or other system perturbations. 

772508 13 
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Specifically, both the NLMS algorithm and the PNLMS algorithm can be made 
robust to large disturbances by a modification of the optimization criterion on which 
these algorithms are based. Recall that the LMS is an iterative algorithm to adjust the 

estimated impulse response so as to minimize the cost function, E{\e^ 2 } 9 the mean square 

error. Each iteration updates the current estimate of h n , by \vc n e n , which is a step in the 

direction of a stochastic approximation to the gradient of E{\en\ 2 }. To make the algorithm 

insensitive to changes of the level of input signal, x n the proportional factor x n is 
normalized, resulting in the NLMS algorithm in (13). It will be recognized by the person 
of skill, utilizing the teachings herein, that other gradient algorithms can be derived by 
changing the cost function to 

J(e n ) = E{q(^)} (29) 
s 

where p(.), and hence J(e^ 9 is any symmetric function with a monotonically 
non-increasing derivative (with respect to its argument). More generally, one can use M- 

\en I 

estimators which are defined as J(e^) = Z nQv — ) • The choice used in (29) makes the 

s 

derivation of the iterative algorithm more consistent with the derivation of the LMS 
algorithm. In any case, the resulting algorithm, analogous to the steepest-descent method 
is 

h n + = h n - //v h J(e n ). (30) 



772508 


14 


BENESTY 2-1-6-13 

The algorithm can be made robust by a proper choice of J(e«), which must be chosen 
such that limen_> J VhJ(e n ) I <-oo. It is known that one might choose VfaJfen) = E{- 
\en\ 

x^ignfe^ y/ ( - — - )s} , where y/ is a hard limiter; 
s 

IK— ) = min{kO,(ti} ? (31) 

s s 

and s is a very important scale factor. The effect of this scale factor, and the manner in 
which it is adapted, is discussed further below. When the Gauss-Newton method is 
utilized to derive the iterative algorithm it is found that the inverse Hessian of the 

2 

minimization criterion (29) ({V/* should be used as a step-size matrix. The Hessian is 
straightforwardly found as 

T I I 

XnX \& n C 

v 2 h J(e„) = E{-^-ip (*—!)}, - 4 R *» ( 32 > 

s s s 

i.e. a constant (<;) times the correlation matrix (R X x) °f the input signal. This result comes 
from our choice of criterion, (12), whose derivative is either zero or one depending on the 
magnitude of e^ s . The inverse of the correlation matrix is far too complex to be used for 
normalization so it is instead preferably approximated as a diagonal matrix, and use a 
stochastic approximation of the gradient is used, leading to the robust NLMS algorithm, 

u \ en | 

h n+I = h n + — xns y/Q — )sign{e»} . (33) 

X n Xn + O S 

The PNLMS algorithm given in (2) can be made robust in an exactly analogous 
manner, yielding the update equation, 
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u \ e » I 
h n +i = h n + — GtXnsy/Q )sign{en}s. (34) 

X„ GnXn + S S 

Alternating the iterations with Gn as given in (24) and (25) yields the robust 
PNLMS++ algorithm. 

The estimate of the scale factor, s 9 should reflect the background noise level at the 
near-end, be robust to shorter burst disturbances (double-talk) and track longer changes of 
the residual error (echo path changes). Therefore it is preferable to chose the scale factor 
estimate as, 

\-X k" | 

s n +i = Asn + sny/Q ), (35) 

fi Sn 

SO = Cx, 

which is very simple to implement. The choice of this method of estimating s is justified, 
as set forth further below. With this choice, the current estimate of s is governed by the 
level of the error signal in the immediate past over a time interval roughly equal to 1/(1 - 
X). When the algorithm has not yet converged, s is large. Hence the limiter is in its linear 
portion and therefore the robust algorithm behaves like the conventional NLMS or 
PNLMS algorithms. When double-talk occurs, the error is determined by the limiter and 
by the scale of the error signal during the recent past of the error signal before the double- 
talk occurs. Thus the divergence rate is reduced for a duration of about 1/(1 - A,). This 
gives ample time for the DTD 30 to act. If there is a system change, the algorithm will 
not track immediately. As the scale estimator tracks the larger error signal the 
nonlinearity is scaled up and convergence rate accelerates. The trade off between 
robustness and tracking rate of the adaptive algorithm supplied by coefficient update 
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device 40 to FIR 10 is thus governed by the tracking rate of the scale estimator which is 
controlled by one single parameter, X. 

As mentioned above, a scale factor, s, can be found by defining the implicit 
function of s given by, 

J 2 (n) = £ ^yM )== o, (36) 
/=o s 

where s is the scale factor, X\ , a forgetting factor. One may chose (•) as, 

X() = KO-A (37) 

because ^(-) is already calculated in (34). The choice of x(') is also not very critical. 
For normalization, fi is chosen such that for a Gaussian process [z eN {0,1 '}], 

E(X(*)} = O. 

This gives 


where 



This choice leads to a scale estimate s = (fy for Gaussian noise upon convergence. 
The purpose of the scale factor is multifold. Traditionally the scale is used to make a 
robust algorithm, e.g. (34), invariant to the background noise level. In this case the noise 
and the echo path to be identified are assumed stationary. In echo cancelling however, it 
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is desirable that the scale factor tracks nonstationary background noise as well as scales 
the nonlinearity appropriately so that the canceller is robust to double-talk bursts. 
Consequently the tracking rate of the scale estimator controls both convergence rate and 
robustness of the algorithm, (34). A recursive scale estimator is derived with a Gauss- 
Newton technique in the same fashion as the R-PNLMS++ algorithm discussed above 
and further below. Thus let 

*n+i = ^-(V^n))'^). (40) 


The gradient of (36) is 


/=0 

M 

Sn 


Cn\ en 

X\— ) + &„-! 


5 2 S 


= --b». (41) 

s 


Let j2(n) = x(- — ') and ^s$( n ) = ^s^2(n). A recursive least squares type of algorithm then 
becomes 


Sn+l = s„+— x{ — } (42) 

bn Sn 


r" r" 

bn+l = Aibn + ^x'i— }. (43) 

Sn Sn 

The complexity of this algorithm is reduced by using the following assumptions: 
V s J(n) is considered stationary and s n converges to the background standard deviation of 
the noise (Gaussian), then 

E{V s ] 2 (n)} = £_;t;-£{Hx'(W)} (44) 

/=o s s 

-> — - — -,"->oo (45) 

S 1 -A\ 


772508 18 


I'll 


§13 

3^ 


BENESTY 2-1-6-13 


<* = ,/-< 


= J-(l-e~^ (46) 


Combining (37), (40) and (45) leads to 


(1-/1 1)5/1 . ,p . .„_. 
s n+ i = s n+ ^ — M— )-/?} (47) 

a Sn 


(,.£0-^^^ (48) 

or or 5n 


/Lyw+ (l-A>A 1 . 5n<y( lfH )> (49) 


m = 1- — (1 - Ai). (50) 


Thus, as mentioned above, R-PNLMS-H- is a robust algorithm that combines R- 
NLMS and R-PNLMS. That is, it uses the same step-size matrix G n as discussed above, 
and the coefficient update equation is 

<pi\en\) = min|e/i|, £«£<)}, (51) 

R ' / Ar+^— — hwi —r~^ — ~G&<^ (52)- 

= U + ^^<p(\en\), (53) 

where !/;(-) is a hardlimiter or Huber function and s n is a scale factor. The preferred 

function ip (|e*|)sign {e^} is shown in Fig. 4. A# and X are constants that control the 

robustness and convergence rate of the algorithm. P is chosen such that if the disturbing 
signal v n is zero-mean white Gaussian noise with variance o v , then s n -*ov, as n -> °o m in 
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practice it may also be useful to bound the minimum value that s n can converge to. This 
value is referred to as s m i n and should preferably be chosen in the same range as o v . 

R-PNLMS++ SIMULATIONS 

Simulations of the above may be considered, by way of non-limiting example, to 
further set forth the preferred embodiments of the echo canceller 1 of the present 
invention and to exemplify the coupling between the DTD 30 and the coefficient vector 
update device 40 utilizing R-PNLMS++. In these exemplary simulations set forth below 
and in the referenced drawing figures, the known Geigel DTD is used. The exact 
coupling and settings of the DTD 30 and adaptive algorithms utilized by coefficient 
vector update device 40 are given in the following as well. In all the simulations, the 
sampling rate is 8 kHz. Two hangover times are used, one for the DTD, Thold = 240 (30 
ms), and one for the scale factor T^old t scale = ^hold The reason for using a longer 
hangover time for the scale factor is to avoid undetected double-talk intervals, which may 
increase the divergence rate of the robust algorithms. Also, when the scale hangover 
counter (Count sca i e ,) is larger than zero, the scale factor is forced to approach a 
prespecified value, s m j n , by the constant ipc- s min$- s min IS an estimate of the standard 
deviation background noise. It has the effect that robustness is maintained if double-talk 
goes on for a longer period of time. The double-talk detection hangover time is controlled 
by the variable Counthold, an exemplary form of calculating same being set for below: 

if l^l^^-maxllx/iljlxn-l,...,!^^- a + 1|}| or Cowh//^/^ >0 
if (Counthold 5 0 ) 
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Counthold = 24 °; 
Count sca i e = 960; 

else 

Counthold = Counthold ' U 
Count sca i e = Count SC ale ' U 
end 


else 


H = 0.2; 

it Count S cale >0 

Count SC ale = Count scale ~ U 
end 

end 

Update eq. (15: R-PNLMS++, 

if Count scale 

i|r n = sign (e n )min{ \e»\ 9 Sn} ; 

ko(\ — X ,^ A ^ 
£n+l =Abi + — ^ — (54) 

else 

*o(l-A 

.Jn+l = Asn H l/fc 

end 

The DTD 30 and scale estimator could share the same hangover counter. 
Estimating the scale s as shown above in equation in (54) instead of (35) reduces 
complexity. Exemplary extra computations for introducing robustness are given in FIG. 5. 
This amounts to less than 0.3% of the total load for a PNLMS++ algorithm with L = 5 12. 

R-PNLMS++ EXPERIMENT 

Experiments were carried out utilizing NLMS, PNLMS and R-PNLMS++ to show 
the performance improvements that may be realized utilizing a preferred robust fast 
converging algorithm such as R-PNLMS++ in coefficient vector update device 40. 

The three algorithms, NLMS, PNLMS++ and R-PNLMS++ are evaluated using 
white Gaussian noise as signals. An example of the performance for a speech signal is 
also shown. General parameter settings are 
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• n = 0.2, L = 512 (64 ms), 6 = 20000, b p = 0.01, p - -0.01 

• o x = 1 900, SNR = 39 dB (echo-to-noise ratio). 

• All three algorithms achieve a minimum mean square error 
of -49 dB. 

• Hybrid attenuation: 20 dB. 

• a Geigel detector assumes 6 dB attenuation. 

• Two sets of parameters for the robust algorithm are shown, 
(X,k(j) = (0.997, 1.1) (0.995, 1.5). These choices result in 
[P * 0.73927, (A = 0.996)]. 

FIG. 6 shows the impulse response, h e p(l), 1 = 0, and the amplitude function of 
the sparse hybrid used in the simulations. Four kinds of experiments are carried out in 
order to illuminate the operation of the R-PNLMS++ versus PNLMS++ and NLMS 
during double-talk, re-convergence and continuous tracking. As performance measures, 
the misalignment and mean square-error (MSE) are used, 


—\x II 2 

Misalignment (n) = 101ogi 0 {L-^ — (55) 

Ml 

{\yn — vn-h n xn ) 
MSE(n) = 101ogio{ J ; -= }, (56) 


(\yn-vn\ > 

The average, < >, is taken over about 1000 samples. This length of averaging is necessary 
in order to reduce the variance in the figures and it does not significantly affect the 
convergence rate in the figures. 
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(i) Double-talk bursts with the DTD either switched off or on 

Half-second bursts of double-talk are introduced at times 3, 6, and 9 seconds of 
levels 30, 6, and 0 dB, respectively, below the far-end speech. FIG. 7 shows the far- and 
near-end signals and the misalignment for parameter setting X = 0.997 and k 0 = 1.1. The 
divergence rates of the algorithms do not strongly depend on the power of the near-end 
signal. Even when the DTD 30 is used as in FIG. 7c, PNLMS++ and NLMS diverge at 
both 6 and 9 s. The DTD 30 does not inhibit adaptation until 122 samples of double-talk 
have entered the algorithms (at 6 s) and this is enough to drive the filters far from their 
optimal setting. R-PNLMS++ is not affected by these samples. A number of detection 
misses between 3 - 3.5 s slightly perturbs the robust estimate. At 9 s, the DTD 30 reacts 
fast at the start of double-talk but makes a detection error only a few samples long after 
the first hangover time. This detection error causes the non robust algorithms to diverge. 
The R-PNLMS++ is not influenced by the detection error occurring between 9 - 9.5 s. 

Results for the parameter setting X = 0.997 and ko = 1.5 are presented in FIG. 8. 
In this case the R-PNLMS++ is less robust as clearly seen at 6 s, in FIG. 8b. Divergence 
due to double-talk detection errors is larger than in the previous case. However, when the 
DTD 30 makes fewer mistakes, the divergence of the R-PNLMS++ is insignificant as 
seen between 9 - 9.5 s. 

An example of the performance during double-talk when speech is used is shown 
in FIGS. 9 and 10. The far-end speaker is female and the near-end speaker is male, and 
the average far- to near-end ratio is 6 dB. R-PNLMS++ performs considerably better 
during double-talk. For the second choice of parameters, (0.995, 1.5), a large number of 
DTD misses eventually affect the hybrid estimate. Observe that the misalignment does 
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not decrease immediately after double-talk stops while the mean square error does. This 
is because some modes of the adaptive filter are poorly excitated by the far-end speech at 
3.5 s. 

(ii) Convergence after abrupt hybrid change 

Three cases of hybrid changes are tested, 

A. hi -hi-2Q0 at $ s, (Practical case). 

B - -hl-200 -> -hl-200 at6s ("Best" case for PNLMS++). 

C hu200 -»M-20Q> '= 0...39, - h^200 I = 40...3U at 9 s, ("Worst" 
PNLMS++). 

The price paid for robustness a ainst double-talk is slower reconvergence aftep 
abrupt hybrid changes which can be seen in FIG. 1 1 . The difference betweeffPNLMS++ 
and R-PNLIMS++ can be made fairly small, FIG. lib. R-JJNLMS++ performs better 
than NLMS in cases A and B for both parameter sgttifigs but is somewhat slower for case 
C. The performance loss for re-convergefice of the robust algorithm is minor. FIGS. 12 
and 13 summarize the divefgence and convergence time of the algorithms where 
divergence/conyer^ence time is defined as the time it takes for the algorithm to 
iQCfetfse/decrease misalignment by 20 dB. 

(iii) Tracking a slowly changing hybrid 
• h/ •* -hi. io gradually between 3 - 4 s. 

Results from tracking a continuously changing hybrid are presented in FIG. 14. 
The hybrid changes slowly over a period of 1 s. R-PNLMS++ and PNLMS++ track 
equally well while NLMS performs poorer. The impact of the non-linearity in this test is 
insignificant. 
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R-PAPA 

As mentioned above, utilization of PLNMS in coefficient vector update device 40 
does not take into account the fact that the speech signal is a correlated signal. Taking 
this into account can further increase convergence, specifically by utilizing APA and 
making it robust. 

Considering all of the above, a robust version of PAPA (and hence of APA) is 
obtained straightforwardly, as set forth below: 

0>(|e«|) = min|e«|,5HA:o}, (57) 

h«+7 = A w + /jC?^^ ^ Me»|)0 sign(e.)l (58) 


(1-*) 

In (58) O denotes elementwise multiplications. 


s n+1 = ^ + LJ4.,4 (59) 


Additionally some of the computational procedures of the Fast Affine Projection 
(FAP) algorithm, , can be incorporated in order to reduce the computational complexity 
of APA and PAPA. The elements of the correlation matrix can be updated recursively. 
Thus, if we define: 

ni,0 = r]l,-J=# (60) 
ri2,0 = 0, (61) 

(62) 

then 

r22,n = >77,w-/ (63) 
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W,n = r Ilfn _]+x 2 n -x 2 n -L (64) 

ril,n = ri2M+ x rFn-l'Xn-LXn-L'l ( 65 ) 
The calculation of the error vector in APA (11) can be approximated as follows, 

*n = d n -x T n h n -i (66) 

e n = [en(l-ii)ein-\] T , (67) 
which reduces the calculations needed by L - 1 multiplies. The complexity of the APA 
algorithm can be reduced further by introducing an alternative coefficient vector, h n 
mentioned in eq. (9). Unfortunately, this cannot be done in PAPA, because invariance of 
the product of the step-size matrix and the excitation vector is destroyed since G n varies 
from one iteration to the next. Shown (Al through A22) below are exemplary fast 
versions of the robust APA and proportional APA. Adaptation is preferably inhibited by 
double-talk detector 30 which also controls the updating of the scale factor s n utilized by 
adaptive scaled non-linearity device 50. In the exemplary simulations discussed further 
herein, a Geigel DTD has been used which is given by, 

if | j>„ | > 9 max{|x„ |, |x„_, |,..., |x„_ L+1 1 } or Counthold > 0 
M = 0; 

if (Counthold * 0) 

Counthold = Thold 
Count sca i e ~ Thold, scale \ 

else 

Counthold = Counthold - 1 
Count sca i e = Count sca \ e -I; 

end 
else 
// = 0.2; 

if Count sca \ e > 0 

Co w/tfscaie = Count -1 ; 
end 
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end 


7hoid is a hangover time for adaptation and 7hold,scale is the hangover time for 
the scale factor. The director threshold, ft = 0.5, if we assume 6 dB of hybrid attenuation 
and ft « 0.71 for a 3 dB assumption. 
Robust APA 


(A5) 


E 0 
r22,n 
ril,n 
ri2,n 


[e,,„e 2 J 7 


0 


ril,n-l 


ril,n-l+x 2 n -x 2 n _ L 


rl 2, n-1 +Xrpc n .i-x n _LX n -L-l 


e n = dn-x T „1i n .l 

=^-^12,n^l,n 
iy(\e n \) = mm{\e„\,k 0 s n } 

e„ = [sign{e„ )y(\e n |)] (l - , f 


E n = 


o 


(All) 

if {Count sca i e < 0) 


(Al) 

(A2) 
(A3) 
(A4) 


(A6) 
(A7) 
(A8) 

(A9) 
(A10) 


else 


(A12) 
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S n + ] = Asn + (l - X)s min (Al 3) 

end 

Robust PAPA 

r22,n = r ll,n-l (A 14) 

m,n = ni,n-l +*l - Xn-L (A15) 

ri2,n = ri2,n-l+XrFn-l -Xn-I*n-L-1 
(A16) 

en = d„-x T „h n .i (A17) 

= min {|en|, snko} (A18) 

e w - [s/gw(en)^(|en|)(l-//)ei,n-if (A19) 

K = h^.xJ/xJ+iXS.XA^ (A20) 
if (Count sca i e < 0) 

^+7 = Aj. + IL^Vd&l) (A21) 


else 


7 = Asn + (1 - A)s min (A22) 


end 


where s m \ n is a preset constant. S m i n is the assumed minimum background power level of 
the noise. Inversion of the correlation matrix R X x,n 1S preferably made with Gaussian 
elimination. Using this technique of the robust APA requires 21+13 multiplies and 3 
divisions, and robust PAPA requires 4L + 12 multiplies and 3 divisions. The individual 
step-size matrix G n is given by (22) to (25) above. 
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R-PAPA SIMULATIONS 

In the following simulations we compare the performance of six algorithms, 
R-PNLMS++, PNLMS++, R-APA, APA, R-PAPA and PAPA. 

The purpose of these simulations is to show the excellent performance of the 
robust algorithms during double-talk and the high convergence rate of the robust PAPA 
and PAPA algorithms when speech is used as excitation signal. With a projection order of 
2, an input signal having the properties of an AR{ 1 } process can be perfectly whitened 
and maximum improvement of the convergence rate is achieved. Speech, however, is not 
an AR{1} process but can be fairly well modeled by an 8th order AR process. An order 
of 2 is preferably chosen as a compromise between complexity and performance. The 
parameter settings chosen for the following simulations are: 

• p = 51 L has been recommended for PNLMS. However, for PNLMS ++ a 
smaller value can be used resulting in lower sensitivity to different 
impulse responses. |i = 0.2, L = 512 (64 ms), 6 = 2.10 5 (R-PNLMS++ and 
PNLMS++), 8=1 10 6 (R-APA, APA, R-PAPA and PAPA), 8 P = 0.01, p 
= 0.001 1 

b x = 1900, SNR= 39 dB (echo-to-noise ratio). 

• Average far-end to double-talk ratio is 6 dB. 

• Hybrid attenuation: 20 dB. 

• Geigel detector assumes 6 dB attenuation. 

Parameters for the robust algorithm are, (A, ko) = (0.997, 1.1). This choice results 
in 0 -,Z~ 0.60665. s m i n = 2, s# = 1000. 

h 0 = 0 

Impulse response and corresponding magnitude function of the test hybrid is 
shown in FIG. 6 
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FIGS. 15-18 show far-end signal, double-talk and the misalignment of the six 
algorithms. The sequence of double-talk starts at different times in the four simulations 
thus showing the reliable performance of the robust algorithm for different double-talk 
situations. The principle of robustness works at all stages of convergence of the 
algorithms. Thus, as seen in FIG. 18, the algorithms (R-PNLMS++, R-APA, R-PAPA) 
resist divergence during double-talk even though they have not yet fully converged. This 
is due to the adaptive scale factor. Initial convergence rates of the PAPA and R-PAPA are 
superior to the other algorithms. FIG. 19 shows the behavior after an abrupt system 
change where the impulse response is shifted 200 samples at 1 second. It is can thus be 
seen that PAPA and R-PAPA are the fastest converging algorithms. Note that the robust 
PAPA converges even faster than PNLMS++. 

R-PAPA EXPERIMENTS - Composite Source Signal as Excitation Signal 

As is known, ITU-T G.168 recommends certain test procedures for evaluating the 
performance of echo cancelers. Test signals used are the so called Composite Source 
Signals (CSS) that have properties similar to those of speech with both voiced and 
unvoiced sequences as well as pauses. This section presents the results from two test 
situations evaluating the performance during double-talk, the so called Test 3A and Test 
3B. In Test 3 A the double-talk sequence is of low level, i.e. 15 dB, below the far-end 
signal. It is of course recognized that a Geigel DTD assuming 6 dB attenuation is not 
able to detect this "double-talk". However, in some cases where the instantaneous far-end 
level is low and the near-end level is high, the DTD may inhibit adaptation. Test 3B 
evaluates the performance of the echo canceller 10 for a high level double-talk sequence. 
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The double-talk level in this case is about the same as that of the far-end signal, thus a 
fixed threshold Geigel DTD assuming 6 dB attenuation is able to detect this double-talk. 
False alarms and failures to detect double-talk are influenced by the chosen threshold in 
the DTD and the attenuation in the hybrid. Three hybrid attenuations are therefore 
evaluated. The parameter and initial value settings for both conducted tests are 

p = 0.2, L = 512 (64 ms), 6 = 2 - 10 5 (R-PNLMS++ and PNLMS++), 6 = 1 - 1 0 8 
(R-APA, APA, R-PAPA and PAPA), 6 p = 0.01, p= 0.001 

Parameters for the robust algorithm are, (A, ko) = (0.9975, 1.5). This choice results 
in p « 0.73927, s m i n = 10, so = 1000. 

b x = 1.3 - 10 4 , SNR 37 dB (echo-to-PCM quantization noise ratio). 
• Hybrid attenuations: 6, 8, 1 1 dB. 

Geigel detect or assumes 3 dB attenuation. 
h<, = 0. 

Multiple tests were carried out using three kinds of hybrids, a sparse, a dispersive 
and a multireflection system. FIG. 20 shows these impulse responses and their magnitude 
functions. Far- and near-end signals (CSS) used in Test 3 A are shown in FIG. 21. 
During the first 625 ms the far-end signal is white noise and the near-end signal is zero. 
This drives the echo canceller 10 to an initial state with zero coefficients and the 
excitation vector is filled with non-zeros samples. The settings of initial variables do not 
influence the convergence rate hence the "true" convergence rate is seen when adaptation 
begins at 625 ms. FIGS. 22 - 26 show that for the parameter settings chosen, the change 
of hybrid attenuation does not seem to influence the performance of the algorithms. 
PAPA and R-PAPA are least sensitive to changes of sparseness of the responses, as 
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recognized by comparing FIG. 22 and 24. These algorithms have superior convergence 
rate compared to the others. All robust algorithms have better average steady-state 
performance (i.e. their average, misalignments are lower) than their respective non-robust 
versions. 

Far- and near-end signals used in Test 3B are shown in Fig. 16. Double-talk starts 
after about 12 seconds. Misalignments of the six algorithms are shown in FIGS. 28 - 32. 
All three robust algorithms handle the double-talk period without any degradation of 
misalignment while non-robust algorithms, despite the Geigel detector, diverge up to 30 
dB. This divergence occurs when the DTD falls to detect the double-talk and as few as 
three samples are enough to drive the hybrid estimate far from optimum. The lengths of 
undetected bursts in these simulations range from a few up to a couple of hundred 
samples. The difference in convergence rate between the algorithms is also more clearly 
seen in this example. 

Thus, based upon the above, the person of skill will recognize that the appropriate 
variables can be adjusted to accommodate differing application needs. For example, to 
increase robustness, one could increase X and decrease k 0 . To increase the speed of 
convergence, decrease X and increase k Q . Moreover, it will also be recognized that the 
present invention will still provide fast convergence and robustness in the event that 
perturbations are caused by changes in the system to which it is applied, such as changes 
in the filter characteristic of the device that the fast impulse response filter is trying to 
emulate and cancel. 

It will be further recognized by those of skill in the art that the system of FIGS. 1- 
3 and/or components thereof can be implemented on a general purpose digital computer 
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using appropriate software to provide updating of the coefficients as shown herein and 
other digital filtering and processing functions. Alternatively, the system or its individual 
components may be implemented on one or more integrated circuits chips having 
software burned or stored therein for providing the functionality herein described. In a 
preferred embodiment, the device is a programmable integrated circuit or a programmable 
logic array. Of course, it will be recognized, that the entire circuit of FIGS. 1-3 or parts 
thereof may also be implemented in a digital signal processor (DSP), that could perform 
all or part of the functions described herein. Moreover, while the inventive system 
described and shown herein has been described in the context of a telephone network, it 
will be recognized by those of skill in the art that such signal processing techniques can 
be applied in other fields of endeavor, such as analog and/or digital audio or video, noise 
cancellation, telephone instruments, conference telephones, radio frequency microwave 
or optical communications, or any other field of endeavor requiring adaptive filtering. 

Thus, while there have been shown and described and pointed out fundamental 
novel features of the invention as applied to preferred embodiments thereof, it will be 
understood that various omissions and substitutions and changes in the form and details 
of the disclosed invention may be made by those skilled in the art without departing from 
the spirit of the invention. It is the intention, therefore, to be limited only as indicated by 
the scope of the claims appended hereto. 
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